-
Notifications
You must be signed in to change notification settings - Fork 1k
improve cua key mapping logic #910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR consolidates key mapping logic for Computer Use Agent (CUA) interactions by extracting duplicate key handling code into a centralized utility function. The changes include:
-
New centralized utility: Creates
lib/agent/utils/cuaKeyMapping.ts
with amapKeyToPlaywright()
function that provides comprehensive key mapping from various formats (Anthropic CUA, OpenAI CUA) to Playwright-compatible key names -
Code deduplication: Removes ~40-45 lines of inline key mapping logic from both
agentHandler.ts
andAnthropicCUAClient.ts
, replacing hardcoded if-else chains with calls to the centralized utility -
File reorganization: Moves
imageCompressionUtils.ts
tolib/agent/utils/imageCompression.ts
to group utilities together in a dedicated utils directory -
Enhanced key support: The new utility provides more comprehensive key mapping including cross-platform variations (e.g., macOS 'COMMAND' vs Windows 'CTRL', 'OPTION' -> 'Alt') and multiple naming conventions for the same keys
The consolidation improves maintainability by centralizing key mapping logic that was previously scattered across multiple files with slightly different implementations. This ensures consistent key handling across all CUA providers and makes it easier to add support for new key mappings in the future. The change fits into the broader Stagehand architecture by creating reusable utilities that can be shared between different agent client implementations (Anthropic, OpenAI, etc.).
Confidence score: 5/5
• This PR is very safe to merge - it's a straightforward refactoring that consolidates duplicate code without changing functionality
• The changes are well-structured, maintain backward compatibility, and improve code organization through proper utility extraction
• All files look correct with proper imports and the key mapping logic appears comprehensive and well-tested
4 files reviewed, no comments
🔄 Feature Parity Issue Created An issue has been automatically created in the Python SDK repository to track parity implementation: |
3319506
to
b20e57e
Compare
.changeset/huge-suns-dress.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we delete this changeset
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
deleted
why
Currently we have a handful of keys that fail to be used, and redundant logic around key handling. It is now consolidated to one location, and properly handles the failing keys
part of STG-587
what changed
consolidated key handling for CUA agents to one util
test plan
tested locally